Scaling Reliably: Improving the Scalability of the Erlang Distributed Actor Platform

Distributed actor languages are an effective means of constructing scalable reliable systems, and the Erlang programming language has a well-established and influential model. While the Erlang model conceptually provides reliable scalability, it has some inherent scalability limits and these force developers to depart from the model at scale. This article establishes the scalability limits of Erlang systems and reports the work of the EU RELEASE project to improve the scalability and understandability of the Erlang reliable distributed actor model. We systematically study the scalability limits of Erlang and then address the issues at the virtual machine, language, and tool levels. More specifically: (1) We have evolved the Erlang virtual machine so that it can work effectively in large-scale single-host multicore and NUMA architectures. We have made important changes and architectural improvements to the widely used Erlang/OTP release. (2) We have designed and implemented Scalable Distributed (SD) Erlang libraries to address language-level scalability issues and provided and validated a set of semantics for the new language constructs. (3) To make large Erlang systems easier to deploy, monitor, and debug, we have developed and made open source releases of five complementary tools, some specific to SD Erlang. Throughout the article we use two case studies to investigate the capabilities of our new technologies and tools: a distributed hash table based Orbit calculation and Ant Colony Optimisation (ACO). Chaos Monkey experiments show that two versions of ACO survive random process failure and hence that SD Erlang preserves the Erlang reliability model. While we report measurements on a range of NUMA and cluster architectures, the key scalability experiments are conducted on the Athos cluster with 256 hosts (6,144 cores). Even for programs with no global recovery data to maintain, SD Erlang partitions the network to reduce network traffic and hence improves performance of the Orbit and ACO benchmarks above 80 hosts. ACO measurements show that maintaining global recovery data dramatically limits scalability; however, scalability is recovered by partitioning the recovery data. We exceed the established scalability limits of distributed Erlang, and do not reach the limits of SD Erlang for these benchmarks at this scal

Early Palaeozoic ocean anoxia and global warming driven by the evolution of shallow burrowing

The evolution of burrowing animals forms a defining event in the history of the Earth. It has been hypothesised that the expansion of seafloor burrowing during the Palaeozoic altered the biogeochemistry of the oceans and atmosphere. However, whilst potential impacts of bioturbation on the individual phosphorus, oxygen and sulphur cycles have been considered, combined effects have not been investigated, leading to major uncertainty over the timing and magnitude of the Earth system response to the evolution of bioturbation. Here we integrate the evolution of bioturbation into the COPSE model of global biogeochemical cycling, and compare quantitative model predictions to multiple geochemical proxies. Our results suggest that the advent of shallow burrowing in the early Cambrian contributed to a global low-oxygen state, which prevailed for ~100 million years. This impact of bioturbation on global biogeochemistry likely affected animal evolution through expanded ocean anoxia, high atmospheric CO2 levels and global warming

Nitrate Reduction Functional Genes and Nitrate Reduction Potentials Persist in Deeper Estuarine Sediments. Why?

Denitrification and dissimilatory nitrate reduction to ammonium (DNRA) are processes occurring simultaneously under oxygen-limited or anaerobic conditions, where both compete for nitrate and organic carbon. Despite their ecological importance, there has been little investigation of how denitrification and DNRA potentials and related functional genes vary vertically with sediment depth. Nitrate reduction potentials measured in sediment depth profiles along the Colne estuary were in the upper range of nitrate reduction rates reported from other sediments and showed the existence of strong decreasing trends both with increasing depth and along the estuary. Denitrification potential decreased along the estuary, decreasing more rapidly with depth towards the estuary mouth. In contrast, DNRA potential increased along the estuary. Significant decreases in copy numbers of 16S rRNA and nitrate reducing genes were observed along the estuary and from surface to deeper sediments. Both metabolic potentials and functional genes persisted at sediment depths where porewater nitrate was absent. Transport of nitrate by bioturbation, based on macrofauna distributions, could only account for the upper 10 cm depth of sediment. A several fold higher combined freeze-lysable KCl-extractable nitrate pool compared to porewater nitrate was detected. We hypothesised that his could be attributed to intracellular nitrate pools from nitrate accumulating microorganisms like Thioploca or Beggiatoa. However, pyrosequencing analysis did not detect any such organisms, leaving other bacteria, microbenthic algae, or foraminiferans which have also been shown to accumulate nitrate, as possible candidates. The importance and bioavailability of a KCl-extractable nitrate sediment pool remains to be tested. The significant variation in the vertical pattern and abundance of the various nitrate reducing genes phylotypes reasonably suggests differences in their activity throughout the sediment column. This raises interesting questions as to what the alternative metabolic roles for the various nitrate reductases could be, analogous to the alternative metabolic roles found for nitrite reductases